Information processing device and information processing method

ABSTRACT

According to an illustrative embodiment, an information processing device is provided. The information processing device includes an image acquisition unit configured to receive an image; a recognition unit configured to acquire a recognition result of a user based on the received image, wherein the recognition result includes a position of the user, the user being associated with a display terminal; an image determination unit configured to determine an object based on the recognition result; and a display control unit configured to control display of the object on the display terminal.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 14/386,120, filed Sep. 18, 2014, which is anational stage application under 35 U.S.C. § 371 of InternationalApplication No. PCT/JP2013/001868, filed Mar. 19, 2013, and claims thebenefit of priority from prior Japanese Patent Application JP2012-071331, filed Mar. 27, 2012, the entire content of which is herebyincorporated by reference. Each of the above-referenced applications ishereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to an information processing device, aninformation processing method, and a program.

BACKGROUND ART

In recent years, techniques for controlling capturing directions havebeen developed. For example, a technique of attaching a light emitteremitting a blinking signal with a predetermined pattern to a subject,specifying a position of the subject based on a detection result whenthe blinking signal having the predetermined pattern is detected, andcontrolling a capturing direction using the position of the subject isdisclosed (e.g., see Patent Literature 1). According to this technique,a captured image is generated to obtain a composition in which thesubject is within the captured image.

In addition to the technique of attaching the light emitter to thesubject, there exist various techniques that generate the captured imageto obtain a composition in which the subject is within the capturedimage.

CITATION LIST Patent Literature

[PTL 1]

JP 2004-153674A

SUMMARY Technical Problem

However, when a user of a portable terminal is to be captured, it isassumed that using only a composition in which the user is within acaptured image is not enough. For example, when the user moves toward adestination, the user is expected to determine a display image havinghigh convenience when the composition is decided after a position of thedestination is also taken into consideration. Accordingly, it isdesirable to realize a technique that causes the user to determine thedisplay image having a higher convenience.

Solution to Problem

In view of the above, the embodiments of the present technology areprovided. According to an illustrative embodiment, an informationprocessing device includes an image acquisition unit configured toreceive an image; a recognition unit configured to acquire a recognitionresult of a user based on the received image, wherein the recognitionresult includes a position of the user, the user being associated with adisplay terminal; an image determination unit configured to determine anobject based on the recognition result; and a display control unitconfigured to control display of the object on the display terminal.

Advantageous Effects of Invention

In accordance with the technology of the present disclosure, it ispossible for a user to determine a display image having higherconvenience.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating a brief overview in accordance with apresent embodiment.

FIG. 2 is a diagram illustrating a configuration of an informationprocessing system in accordance with the present embodiment.

FIG. 3 is a block diagram illustrating a functional configuration of aninformation processing device in accordance with the embodiment.

FIG. 4 is a diagram illustrating a brief overview of a userspecification function of an information processing device.

FIG. 5 is a diagram illustrating a display example of a portableterminal when the user specification fails.

FIG. 6 is a diagram illustrating a display example of a portableterminal when the user specification succeeds.

FIG. 7 is a diagram illustrating a display example of a portableterminal when the user specification fails.

FIG. 8 is a diagram illustrating a display example of a portableterminal when the user specification fails.

FIG. 9 is a diagram illustrating a display example of a portableterminal when the user specification fails.

FIG. 10 is a diagram illustrating a brief overview of a compositiondecision function of the information processing device.

FIG. 11 is a diagram illustrating a display image at the time of settinga first mode.

FIG. 12 is a diagram illustrating a display image at the time of settinga second mode.

FIG. 13 is a diagram illustrating a variation of a technique ofdetermining a display image.

FIG. 14 is a diagram illustrating an example of the determined displayimage.

FIG. 15 is a flowchart illustrating a flow of composition decisionoperations of the information processing device.

FIG. 16 is a diagram illustrating an example of a depiction changefunction of the information processing device.

FIG. 17 is a diagram illustrating another example of a depiction changefunction of the information processing device.

FIG. 18 is a diagram illustrating another example of a depiction changefunction of the information processing device.

FIG. 19 is a diagram illustrating another example of a depiction changefunction of the information processing device.

FIG. 20 is a diagram illustrating another example of a depiction changefunction of the information processing device.

FIG. 21 is a diagram illustrating another example of a depiction changefunction of the information processing device.

FIG. 22 is a diagram illustrating another example of a depiction changefunction of the information processing device.

FIG. 23 is a flowchart illustrating a flow of depiction changeoperations of the information processing device.

DESCRIPTION OF EMBODIMENTS

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

Further, in this specification and the drawings, each of a plurality ofstructural elements having substantially the same function isdistinguished by affixing a different letter to the same referencenumeral in some cases. However, when there is no particular necessity todistinguish between structural elements having substantially the samefunction, they are denoted by the same reference numeral.

First, “Description of Embodiments” will be described in accordance withthe order listed below.

1. User specification function

2. Composition decision function

3. Depiction change function

4. Conclusion

Functions of the information processing device according to the presentembodiment are broadly classified into a user specification function, acomposition specification function, and a depiction change function.First, before describing each of the functions, a condition that is apremise of each function will be described.

FIG. 1 is a diagram illustrating a brief overview of the presentembodiment. As shown in FIG. 1, a user U having a portable terminal 20is present in a real space. The user U may be present anywhere in thereal space. In addition, a function of displaying an image is installedin the portable terminal 20, and it is possible for the user U to view ascreen displayed by the portable terminal 20. The user U is captured byan imaging device 30 provided to be separate from the portable terminal20. A position in which the imaging device 30 is to be provided is notparticularly limited. For example, the imaging device may be provided ina building such as in a monitoring camera, or may be provided in amobile object such as a vehicle. In addition, a kind of the imagingdevice 30 is not particularly limited, but may be an infrared camera orthe like, for example.

FIG. 2 is a diagram illustrating a configuration of an informationprocessing system in accordance with the present embodiment. As shown inFIG. 2, the information processing system 1 includes an informationprocessing device 10, a portable terminal 20, and an imaging device 30.Although the information processing device 10 and the portable terminal20 are connected via a network 40 in FIG. 2, the number of the portableterminal 20 connected to the network 40 may not be one. For example, aplurality of the portable terminals 20 may be present, and each of theportable terminals 20 may be connected to the network 40.

In addition, as shown in FIG. 2, the information processing device 10 isconnected to the imaging device 30. Although one imaging device 30 isconnected to the information processing device 10 in the example shownin FIG. 2, the number of the imaging device 30 may not be one. Forexample, a plurality of the imaging devices 30 may be present, and eachof the imaging devices 30 may be connected to the information processingdevice 10.

In addition, although the information processing device 10 is configuredto be separate from the portable terminal 20 and the imaging device 30in the example shown in FIG. 2, the information processing device may beembedded in the portable terminal 20, or may be embedded in the imagingdevice 30. A captured image captured by the imaging device 30 isprovided to the portable terminal 20 via the information processingdevice 10, and it is possible for the portable terminal 20 to displaythe captured image. The information processing device 10 may use thecaptured image provided from the imaging device 30 to exert the userspecification function, the composition decision function, and thedepiction change function.

FIG. 3 is a block diagram illustrating functional configuration of theinformation processing device 10 in accordance with the presentembodiment. As shown in FIG. 3, the information processing device 10includes a processing control unit 100, a display control unit 160, anda storage unit 50. In addition, the processing control unit 100 has animage acquisition unit 111, a parameter acquisition unit 112, acandidate extraction unit I13, a specification unit 114, and anauthentication unit 115. The image acquisition unit 111, the parameteracquisition unit 112, the candidate extraction unit 113, thespecification unit 114, and the authentication unit 115 are blocksmainly associated with the user specification function of theinformation processing device 10.

In addition, the processing control unit 100 has a recognition unit 121,a condition decision unit 122, an information addition unit 123, a modesetting unit 124, a composition decision unit 125, and an imagedetermination unit 126. The recognition unit 121, the condition decisionunit 122, the information addition unit 123, the mode setting unit 124,the composition decision unit 125, and the image determination unit 126are blocks mainly associated with the composition decision function ofthe information processing device 10.

In addition, the processing control unit 100 has a change detection unit131 and a depiction change unit 132. The change detection unit 131 andthe depiction change unit 132 are blocks mainly associated with thedepiction change function of the information processing device 10.

The processing control unit 100 and the display control unit 160correspond to processors such as a CPU (Central Processing Unit) or aDSP (Digital Signal Processor). The processing control unit 100 and thedisplay control unit 160 exert various functions of the processingcontrol unit 100 and the display control unit 160 by executing a programstored in the storage unit 50 or another storage medium.

The storage unit 50 uses a storage medium such as a semiconductor memoryor a hard disk to store programs and data for processes to be carriedout by the processing control unit 100 and the display control unit 160.For example, the storage unit 50 stores a feature amount dictionary usedto recognize an object. Further, the storage unit 50 may store arecognition result that is generated as a result of the objectrecognition. Although the storage unit 50 is configured to be embeddedin the information processing device 10 in the example shown in FIG. 3,the storage unit 50 may be configured to be separate from theinformation processing device 10.

Premises of the respective functions have been described. Hereinafter,each of the functions will be described in order.

1. USER SPECIFICATION FUNCTION

First, the user specification function of the information processingdevice 10 will be described. FIG. 4 is a diagram illustrating a briefoverview of the user specification function of the informationprocessing device 10. The image acquisition unit 111 first acquires acaptured image captured by the imaging device 30. In the example shownin FIG. 4, as an example of the captured image, a captured image Img inwhich a real space shown in FIG. 1 is captured is acquired by the imageacquisition unit 111. The captured image acquired by the imageacquisition unit 111 is controlled to be displayed in the portableterminal 20 by the display control unit 160.

According to the user specification function, a user is specified from asubject shown in the captured image acquired by the image acquisitionunit 111. The purpose of the user specification is not particularlylimited. However, it is assumed herein that a network service isprovided by the information processing device 10 and the userspecification is carried out so as to allow the user U to log into thenetwork service via the portable terminal 20.

In addition, the kind of the network service is not particularlylimited. However, a service allowing a user to write a message or toread the message written by the user or another user using a portableterminal in which logging into the network service is completed isassumed herein. Although the timing for specifying the user is notparticularly limited, a timing at which an operation for the user torequest the user specification is performed on the portable terminal 20may be used. The operation for requesting the user specification may bean operation for request to log into the network service.

In order to specify the user from the subject shown in the capturedimage, the parameter acquisition unit 112 acquires parameters associatedwith the user to be specified. Although the timing for the parameteracquisition unit 112 to acquire the parameters is not particularlylimited, for example, the parameters may be acquired in advance from theportable terminal 20 of the user, or may be acquired from the portableterminal 20 when the user is specified.

The kind of the parameter to be acquired by the parameter acquisitionunit 112 is not particularly limited. For example, as shown in FIG. 4,the parameter acquisition unit 112 may acquire a face image of the useras the parameter P1, and may acquire sensor data representing a movementof the user as the parameter P2. A device capturing the face image ofthe user is not particularly limited. For example, the face image of theuser may be a face image captured by the portable terminal 20, or may bea face image captured by another device.

In addition, a sensor detecting the movement of the user is notparticularly limited. The sensor detecting the movement of the user maybe an acceleration sensor, a sensor receiving GPS signals, and a sensorreceiving radio waves transmitted from a Wi-Fi base station. Inaddition, the sensor detecting the movement of the user may be embeddedin the portable terminal 20, and may be mounted on a body of the user.

Before the user is specified by the specification unit 114, a candidateof the user may be specified by the candidate extraction unit 113 (stepS11). For example, when coordinates on the captured image displayed inthe portable terminal 20 by the user operation input to the portableterminal 20 (e.g., a touch operation of the touch panel) are designated,the parameter acquisition unit 112 acquires the coordinates, and thecandidate extraction unit 113 may specify a subject selected by thecoordinates (e.g., the subject present on the coordinates) as thecandidate of the user. By specifying the candidate of the user, a loadnecessary to specify the user is reduced.

The specification unit 114 tries to specify the user from the capturedimage based on the parameter acquired by the parameter acquisition unit112 (step S12). For example, when the face image of the user is acquiredas the parameter P1 by the parameter acquisition unit 112, thespecification unit 114 may specify the subject extracted based on acombination of a facial region of the subject shown in the capturedimage acquired by the image acquisition unit 111 and the face image asthe user. For example, the specification unit 114 may specify thesubject having the face image of the user and the facial region of thesimilarity exceeding a threshold value as the user.

In addition, for example, when sensor data representing the movement ofthe user is acquired as the parameter P2 by the parameter acquisitionunit 112, the specification unit 114 may specify the subject extractedbased on a combination of the movement of the subject detected from thecaptured image acquired by the image acquisition unit 111 and the sensordata as the user. For example, the specification unit 114 may specifythe subject in which the movement of the user in the captured image andthe sensor data of the similarity exceeding a threshold value aredetected as the user. In addition, when the candidate of the user isspecified, the specification unit 114 may specify the user based on theparameter from the candidate of the user.

As a result of the specification unit 114 trying to specify the user, itis also assumed that the user is not specified. For example, a case inwhich the similarity does not exceed the threshold value corresponds tothe situation assumed above. Accordingly, when the user is not specifiedby the specification unit 114, the display control unit 160 controls theportable terminal 20 to display information indicating the operation ofthe user necessary to specify the user in the portable terminal 20. Inaccordance with the control of the display control unit 160, theinformation indicating the movement of the user is displayed in theportable terminal 20 (step S13).

FIG. 5 is a diagram illustrating a display example of the portableterminal 20 when the user specification fails. For example, when theuser is not specified by the specification unit 114, the display controlunit 160 may control the portable terminal 20 such that the informationprompting the user to turn his or her face toward the imaging device isdisplayed in the portable terminal 20. In FIG. 5, the example in whichthe imaging device 30 is assumed to be a monitoring camera and thedisplay control unit 160 controls the portable terminal 20 such that amessage M1 indicating “Please turn your face toward the monitoringcamera” is displayed in the portable terminal 20 is shown.

A description will be made with reference back to FIG. 4. When the useris specified by the specification unit 114, before the flow proceeds tostep S15, the authentication unit 115 may perform the authenticationprocess on the portable terminal 20 specified by the specification unit114 (step S14). In particular, for example, as shown in FIG. 4, when apassword is input by a user operation input to the portable terminal 20,the authentication unit 115 may determine success and failure bydetermining whether the input password matches a password that isregistered in advance. When the authentication is successful, the flowproceeds to step S15. When the authentication fails, the authenticationprocess may be performed again instead of proceeding to step S15.

When the flow proceeds to step S15, for example, logging into thenetwork service is completed (step S15) and the display control unit 160controls the portable terminal 20 such that the captured image and theadditional information are displayed by the portable terminal 20. Inaccordance with the control of the display control unit 160, thecaptured image and the additional information are displayed in theportable terminal 20 (step S16). Although the additional information isnot particularly limited, for example, the additional information may bea message written by the user himself/herself or another user.

FIG. 6 is a diagram illustrating a display example of the portableterminal 20 when the user specification is successful. For example, whenthe user is specified by the specification unit 114, the display controlunit 160 controls the portable terminal 20 such that the captured imageand the additional information are displayed. An example in which thedisplay control unit 160 controls the portable terminal 20 such that amessage written by another user is displayed as the additionalinformation is shown in FIG. 6.

In addition, the information prompting the user to turn his or her facetoward the imaging device is not limited to the example shown in FIG. 5.FIG. 7 is a diagram illustrating a display example of the portableterminal 20 when the user specification fails. The display control unit160 may control the portable terminal 20 such that informationindicating the movement of the user necessary to specify the user isdisplayed in the portable terminal 20 based on a position of thecandidate of the user extracted by the candidate extraction unit 113 anda position of the imaging device 30 that is set in advance.

For example, as shown in FIG. 7, the display control unit 160 maycontrol the portable terminal 20 such that the portable terminal 20displays a straight line in a direction that the candidate of the userfaces and a straight line in a direction in which the imaging device 30is present using the position of the candidate of the user as areference. In addition, as shown in FIG. 7, the display control unit 160may control the portable terminal 20 such that an arrow representing arotational direction from the straight line in the direction that thecandidate of the user faces to the straight line in the direction inwhich the imaging device 30 is present is displayed in the portableterminal 20. In addition, as shown in FIG. 7, the display control unit160 may control the portable terminal 20 such that a message M2 of “TurnLeft” is displayed as the message representing the rotational directionthat the user has seen in the portable terminal 20.

In addition, when the user is not specified by the specification unit114, a technique of specifying the user may be switched. FIG. 8 is adiagram illustrating a display example of the portable terminal 20 whenthe user specification fails. When the user is not specified by thespecification unit 114, the display control unit 160 may control theportable terminal 20 such that information indicating that the user isspecified by another technique is displayed in the portable terminal 20.For example, the display control unit 160 may control the portableterminal 20 such that a message M3 of “a technique of specifying theuser is switched” is displayed as the information indicating that theuser is specified by another technique in the portable terminal 20. Thespecification unit 114 may specify the subject extracted by anotherspecification technique as the user.

For example, the specification unit 114 tries to specify the subjectextracted based on a combination of the face region of the subject shownin the captured image and the face image of the user as the user. Inthis case, when the user is not specified, the specification unit 114may try to specify the subject extracted based on a combination of themovement of the subject detected by the captured image and sensor dataas the user. In addition, the user specification may be tried in reverseorder.

In addition, when the user is not specified by the specification unit114, the user may be prompted to cause a difference between movements ofanother subject shown in the captured image and the user to be clear.Accordingly, when a subject moving out of the defined normal range isdetected from the captured image, it is possible for the specificationunit 114 to specify the subject as the user.

FIG. 9 is a diagram illustrating a display example of the portableterminal 20 when the user specification is failed. For example, when theuser is not specified by the specification unit 114, the display controlunit 160 may control the portable terminal 20 such that informationprompting the user to move the portable terminal 20 is displayed asinformation indicating the movement of the user necessary to specify theuser in the portable terminal 20. For example, as shown in FIG. 9, thedisplay control unit 160 may control the portable terminal 20 such thata message M4 of “Please move your portable terminal” is displayed as theinformation prompting the user to move the portable terminal 20 in theportable terminal 20.

As described above, according to the user specification function of theinformation processing device 10, user specification is tried from thecaptured image based on the parameter associated with the user of theportable terminal, and the display control unit 160 controls theportable terminal 20 such that information indicating the movement ofthe user necessary to specify the user is displayed in the portableterminal 20 when the user is not specified. Therefore, the user isexpected to be easily specified from the captured image.

2. COMPOSITION DECISION FUNCTION

Next, the composition decision function of the information processingdevice 10 will be described. FIG. 10 is a diagram illustrating a briefoverview of the composition decision function of the informationprocessing device 10. As described above, when the capture image iscaptured by the imaging device 30, the captured image is displayed inthe portable terminal 20 of the user. It is possible for the user toview the captured image displayed in the portable terminal 20. A scenein which the user moves toward the destination while viewing thecaptured image is assumed herein. Decision on a preferred composition asthe captured image viewed by the user will be described.

First, information indicating the destination of the user is assumed.The information indicating the destination of the user may be input bythe user operation input to the portable terminal 20, and may be set byan application run by the portable terminal 20. For example, theinformation indicating the destination of the user is depicted at aposition in the real space. The information indicating the destinationof the user is acquired by the information processing device 10. Thecomposition decision unit 125 decides the composition based on theinformation indicating the destination of the user and the recognitionresult of the user based on the captured image acquired by the imageacquisition unit 111. The composition is a diagram illustrating an idealposition of an object in the image (or an ideal direction of theobject).

The recognition result of the user includes at least the user positionin the real space and the travelling direction of the user. For example,the recognition result of the user is acquired by the recognition unit121. In other words, the recognition unit 121 obtains the recognitionresult by recognizing the user based on the captured image. In theexample shown in FIG. 10, the user position in the real space is denotedas Pu, the travelling direction of the user is denoted as V1, and thedirection of the destination of the user using the user position Pu as areference is denoted as V2.

For example, when the user position is recognized as the recognitionresult, the composition decision unit 125 may decide the compositionbased on the direction of the destination of the user using the userposition as a reference. For example, in the example shown in FIG. 10,the composition decision unit 125 may decide the composition based onthe direction V2 of the destination of the user using the user positionPu as the reference. For example, when the composition is decided basedon V2, the composition decision unit 125 may decide the image in which aregion in the direction V2 using the user position Pu as the referenceis included to be larger than a region in a direction opposite to thedirection V2 using the user position Pu as the reference as thecomposition. An example of the composition as decided above is shown asK2 in FIG. 10. The point C2 indicates a middle point of the compositionK2.

In addition, for example, when the travelling direction of the user isrecognized as the recognition result, the composition decision unit 125may decide the composition based on the travelling direction of theuser. For example, in the example shown in FIG. 10, the compositiondecision unit 125 may decide the composition based on the travellingdirection V1 of the user. For example, when the composition is decidedbased on V1, the composition decision unit 125 may decide the image inwhich a region in the direction V1 using the user position Pu as thereference is included to be larger than a region in a direction oppositeto the direction V1 using the user position Pu as the reference as thecomposition. An example of the composition as decided above is shown asK1 in FIG. 10. The point C1 indicates a middle point of the compositionK1.

The composition decision unit 125 may uniformly decide the compositionbased on the travelling direction V1 of the user, may uniformly decidethe composition based on the direction V2 of the destination of the userusing the user position Pu as the reference, or may selectively use anyof the decision techniques mentioned above. In other words, when a modein which the composition is decided based on the travelling direction V1of the user is a first mode and a mode in which the composition isdecided based on the direction V2 of the destination of the user usingthe user position Pu as the reference is a second mode, the mode settingunit 124 may set any of these two modes. In this case, the compositiondecision unit 125 may decide the composition in accordance with the modeset by the mode setting unit 124. The mode may be set based on the useroperation input to the portable terminal 20, or may be set by anapplication run by the portable terminal 20.

The image determination unit 126 determines the display image based onthe composition decided by the composition decision unit 125. Forexample, the image determination unit 126 may determine an image inconformity with the composition decided by the composition decision unit125 as the display image, or may determine an image closest to thecomposition decided by the composition decision unit 125 within theselectable range as the display image. The display control unit 160controls the portable terminal 20 such that the display image determinedby the image determination unit 126 is displayed in the portableterminal 20.

FIG. 11 is a diagram illustrating a display image at the time of settingthe first mode. For example, when the captured image Img is acquired asshown in FIG. 10 and the first mode is set by the mode setting unit 124,the composition K1 is decided based on the travelling direction V1 ofthe user. In this case, for example, as shown in FIG. 11, the imagedetermination unit 126 may determine an image in conformity with thecomposition K1 as the display image D1. In addition, the compositiondecision unit 125 may decide the composition in which the user ispresent in the middle. As shown in FIG. 11, the composition in which theuser is present at a position spaced a predetermined distance d1 fromthe middle may be decided.

In addition, a current capturing condition may be decided. In otherwords, the condition decision unit 122 may decide whether or not thecondition for capturing the user is prepared, and the informationaddition unit 123 may add information indicating that the condition isnot prepared to the display image when the condition decision unit 122decides that the condition is not prepared. For example, when a regioncapable of being captured is decided by the condition decision unit 122,the information addition unit 123 may add an object indicating aboundary of the region capable of being captured to the display image.

In addition, when the boundary of the region capable of being capturedin the travelling direction V1 of the user using the user position Pu asthe reference is decided by the condition decision unit 122, theinformation addition unit 123 may add an object indicating the boundaryof the region capable of being captured to the display image. Since theboundary of the region capable of being captured in the travellingdirection V1 of the user using the user position Pu as the reference hasbeen decided in FIG. 11, the object indicating the boundary of theregion capable of being captured is added to the display image D1 as A1.Alternatively, when the condition decision unit 122 decides that adistance between the user position Pu and the boundary of the regioncapable of being captured is shorter than a threshold, the informationaddition unit 123 may add the object indicating the boundary of theregion capable of being captured to the display image.

In addition, when the condition decision unit 122 decides that thebrightness of the captured image is less than a threshold, theinformation addition unit 123 may add an object indicating deteriorationof the capturing condition to the display image. Although prompting theuser to change the travelling direction is possible by virtue of such aprocess, the information provided to the user is not limited to theadded object. For example, the information addition unit 123 may controlthe portable terminal 20 to vibrate, and may control the portableterminal 20 to generate an alarm tone.

FIG. 12 is a diagram illustrating a display image at the time of settingthe second mode. For example, when the captured image Img is acquired asshown in FIG. 10 and the second mode is set by the mode setting unit124, the composition K2 is decided based on the direction V2 of thedestination of the user using the user position Pu as the reference. Inthis case, for example, as shown in FIG. 12, the image determinationunit 126 may determine an image in conformity with the composition K2 asthe display image D2. In addition, although the composition decisionunit 125 decides the composition in which the user is present in themiddle, the composition decision unit may decide the composition inwhich the user is present at a position spaced a predetermined distanced2 from the middle as shown in FIG. 12.

So far, the image determination unit 126 has used the technique ofdetermining the display image by cutting out a portion of the capturedimage captured by the imaging device 30 based on the composition decidedby the composition decision unit 125. When the imaging device 30 is onefixed camera, only the technique mentioned above is employed. However,when the imaging device 30 is a direction adjustable camera or anangle-of-view adjustable camera, or when the imaging device 30 isprovided in a plural number, other techniques may be employed.

In this case, the fixed camera means a camera in which the capturingdirection or the angle of view is not adjustable. The directionadjustable camera means a movable camera, and may be a camera having apan function or a camera having a tilt function. The angle-of-viewadjustable camera means a camera capable of changing the angle of view.FIG. 13 is a diagram illustrating a variation of a technique ofdetermining the display image. As shown in FIG. 13, when the capturingcamera 30 is the fixed camera, the image determination unit 126 employsthe technique of determining the region selected from the captured imageas the display image.

In addition, as shown in FIG. 13, when the imaging device 30 is thedirection adjustable camera, the image determination unit 126 determinesthe capturing direction of the imaging device 30 that provides thecaptured image, and determines the captured image captured in thedetermined capturing direction as the display image. For example, theimage determination unit 126 determines the capturing direction of theimaging device 30 based on the composition decided by the compositiondecision unit 125. In particular, the image determination unit 126 maydetermine the direction in which an image closest to the compositiondecided by the composition decision unit 125 can be captured as thecapturing direction of the imaging device 30.

In addition, as shown in FIG. 13, when the imaging device 30 is theangle-of-view adjustable camera, the image determination unit 126determines the angle of view of the imaging device 30 that provides thecaptured image, and determines the captured image captured at thedetermined angle of view as the display image. For example, the imagedetermination unit 126 determines the angle of view of the imagingdevice 30 based on the composition decided by the composition decisionunit 125. In particular, the image determination unit 126 may determinethe angle of view at which an image closest to the composition decidedby the composition decision unit 125 can be captured as the angle ofview of the imaging device 30.

In addition, as shown in FIG. 13, when captured images provided from therespective imaging devices 30 are acquired by the image acquisition unit111, the image determination unit 126 determines the captured imageselected from the captured images provided from the respective imagingdevices 30 as the display image. For example, the image determinationunit 126 determines the captured image selected from the captured imagesprovided from the respective imaging devices 30 as the display imagebased on the composition decided by the composition decision unit 125.In particular, the image determination unit 126 may select the capturedimage closest to the composition decided by the composition decisionunit 125 from the captured images provided from the respective imagingdevices 30, and may determine the selected captured image as the displayimage.

In addition, in the example described above, the image closest to thecomposition decided by the composition decision unit 125 is used as thedisplay image. However, when only an image having a difference exceedinga threshold between the image and the composition is acquired, thedisplay image may not be determined. In this case, it is also assumedthat it is difficult for the image determination unit 126 to determinethe display image based on the composition decided by the compositiondecision unit 125. Accordingly, in such a case, the image determinationunit 126 may determine the display image based on detection data otherthan the composition.

The detection data is not particularly limited. For example, it ispossible for the image determination unit 126 to determine an image inwhich a position closest to the user position specified by thespecification unit 114 based on the sensor data is captured as thedisplay image. In addition, the image determination unit 126 maydetermine an image having the smallest number of persons shown in theimage as the display image, or may determine an image having thesmallest density of persons shown in the image as the display image.Further, the image determination unit 126 may determine an image inwhich the largest range is captured as the display image, or maydetermine the captured image that is most frequently used as the displayimage.

In addition, when the imaging device 30 is not present or the like, itis also assumed that the captured image is not acquired by the imageacquisition unit 111. In such a case, the image determination unit 126may determine an image according to information indicating thedestination of the user and the user position detected based on dataother than the captured image as the display image.

For example, when position information of the user is acquired based onthe signal received by a sensor receiving GPS signals, the imagedetermination unit 126 may use the position information and informationindicating the destination of the user to search an image from a webpage or the like. The image may be a plotted map, or may be a capturedphotograph. In addition, when plural images are acquired by searching,the image determination unit 126 may determine a new image generated bysynthesizing a plurality of images as the display image.

FIG. 14 is a diagram illustrating an example of the determined displayimage. As shown in FIG. 14, the display control unit 160 may add anobject Obj1 indicating a distance from the user position Pu as areference to the destination of the user, an object Obj2 indicating anestimated time of arriving in the destination and a distance to thedestination, and so forth, to the display image D2.

In addition, as shown in FIG. 14, the display control unit 160 may addan object Obj3 indicating a bearing, an object Obj4 indicating adirection of the destination of the user using the user position Pu asthe reference, and so forth, to the display image D2. In addition, asshown in FIG. 14, the display control unit 160 may add an object Obj5indicating a direction of the destination of the user using the userposition Pu as the reference to the user position Pu and may add anobject Obj6 indicating the user position Pu to a position of the user inthe display image D2.

Next, a flow of the composition decision operation of the informationprocessing device 10 will be described. FIG. 15 is a flowchartillustrating the flow of the composition decision operation of theinformation processing device 10. As shown in FIG. 15, when the capturedimage is not acquired by the image acquisition unit 111 (“No” in stepS21), the recognition unit 121 recognizes the user from data other thanthe captured image (step S22), and the flow proceeds to step S26.

On the other hand, when the captured image is acquired by the imageacquisition unit 111 (“Yes” in step S21), the recognition unit 121recognizes the user from the captured image (step S23), and thecondition decision unit 122 decides whether or not a condition forcapturing is prepared (step S24). When the condition decision unit 122decides that the condition for capturing is prepared (“Yes” in stepS24), the flow proceeds to step S26. On the other hand, when thecondition decision unit 122 decides that the condition for capturing isnot prepared (“No” in step S24), the information addition unit 123 addsinformation indicating that the condition is not prepared to thecaptured image (step S25), and the flow proceeds to step S26.

When the flow proceeds to step S26, the mode setting unit 124 sets anyone of the first mode and the second mode (step S26), and thecomposition decision unit 125 decides the composition in accordance withthe mode set by the mode setting unit 124 based on the recognitionresult of the recognition unit 121 (step S27). The image determinationunit 126 then tries to determine the display image based on thecomposition decided by the composition decision unit 125 (step S28).

In this case, when the display image is determined by the imagedetermination unit 126 (“Yes” in step S29), the flow proceeds to stepS31. On the other hand, when the display image is not determined by theimage determination unit 126 (“No” in step S29), the image determinationunit 126 determines the display image based on detection data other thanthe composition (step S30), and the flow proceeds to step S31. Thedisplay control unit 160 controls the portable terminal 20 such that thedisplay image determined by the image determination unit 126 isdisplayed in the portable terminal 20 (step S31), and the flow isfinished.

As described above, according to the composition decision function ofthe information processing device 10, the composition is decided basedon information indicating the destination of the user and therecognition result of the user based on the captured image. In addition,the display image is determined based on the determined composition. Inaccordance with such control, the display image having higherconvenience is expected to be determined for the user.

3. DEPICTION CHANGE FUNCTION

Next, the depiction change function of the information processing device10 will be described. As described above, it is assumed that theinformation processing device 10 provides a network service and thenetwork service is provided to the portable terminal 20 when the user Ulogs into the network service via the portable terminal 20.

Logging into the network service is not limited to the user U, and maybe performed by another user. In other words, there is a possibilitythat a person shown in the captured image (hereinafter, also referred toas a subject) enjoy the network service. Hereinafter, a function ofeasily finding the change in state in the network service of the subjectshown in the captured image will be described. In addition, the subjectshown in the captured image may be recognized by the recognition unit121 using the same technique as the technique specified by the user.

The change detection unit 131 detects the change in state in the networkservice of the subject recognized from the captured image acquired bythe image acquisition unit 111. The state in the network service is notparticularly limited. For example, the state may be informationindicating whether or not the portable terminal of the subject logs intothe network service. In other words, when the change detection unit 131detects that the portable terminal of the subject has logged into thenetwork service, the depiction change unit 132 may change the depictionof the subject shown in the captured image.

In addition, the state in the network service may be informationindicating whether or not a specific process is carried out while beinglogged into the network service. In other words, when the changedetection unit 131 detects that the movement in the network service ofthe portable terminal of the subject satisfies a predeterminedcondition, the depiction change unit 132 may change the depiction of thesubject shown in the captured image. The predetermined condition is notparticularly limited. For example, the predetermined condition may be acondition that a message was written, and may be a condition that themessage was written within a predetermined time.

When the change in state is detected by the change detection unit 131,the depiction change unit 132 changes the depiction of the subject shownin the captured image. A region of the subject shown in the capturedimage may be figured out in any way by the depiction change unit 132.For example, when a difference between the captured image in which thesubject is not shown and the captured image in which the subject isshown occurs, the differential region therebetween may be found as theregion of the subject. In addition, a method of changing the depictionof the subject is not particularly limited.

Further, when the user is specified by the specification unit 114, thedepiction change unit 132 may change the depiction of the user byemphasizing the user shown in the captured image. In addition, a methodof emphasizing the user is not particularly limited. The display controlunit 160 controls the portable terminal 20 such that the display imagethat may be obtained by causing the depiction change unit 132 to changethe depiction is displayed in the portable terminal 20.

Techniques of emphasizing the user will be described with reference todrawings. FIG. 16 is a diagram illustrating an example of the depictionchange function of the information processing device 10. As shown inFIG. 16, the depiction change unit 132 may emphasize the user U bydecreasing the definition of subjects other than the user U in thecaptured image.

FIG. 17 is a diagram illustrating another example of the depictionchange function of the information processing device 10. As shown inFIG. 17, the depiction change unit 132 may emphasize the user U byemphasizing an outline L1 of the user U in the captured image. Forexample, the user U may be emphasized by making the outline L1 of theuser U thick in the captured image.

FIG. 18 is a diagram illustrating another example of the depictionchange function of the information processing device 10. As shown inFIG. 18, the depiction change unit 132 may emphasize the user U bychanging the pixel value of subjects other than the user U in thecaptured image. For example, the user U may be emphasized by decreasingthe brightness of subjects other than the user U in the captured image.

FIG. 19 is a diagram illustrating another example of the depictionchange function of the information processing device 10. As shown inFIG. 19, the depiction change unit 132 may emphasize the user U suchthat the user U is zoomed in on and displayed in the captured image. Inaddition, as shown in FIG. 19, the depiction change unit 132 may zoomout and display the user U after the zoom-in display is carried out.

FIG. 20 is a diagram illustrating another example of the depictionchange function of the information processing device 10. The depictionchange unit 132 may emphasize the user by displaying an object at aposition of the user or a position near the user in the captured image.The range of surrounding the user may be defined in advance. In theexample shown in FIG. 20, the depiction change unit 132 emphasizes theuser U by adding objects Obj8 and Obj9 to the position around the userin the captured image.

In addition, the degree of emphasizing the user U may be constant, ormay be changed depending on the situation. For example, the depictionchange unit 132 may change the degree of emphasizing the user inaccordance with the size of the user in the captured image. FIG. 21 is adiagram illustrating another example of the depiction change function ofthe information processing device 10. For example, the depiction changeunit 132 may increase the degree of emphasizing the user to the sameextent to which the size of the user in the captured image decreases.

In the example shown in FIG. 21, as the position of the user U isfarther from the imaging device 30, the size of the user U in thecaptured image is smaller. Therefore, the depiction change unit 132increases the degree of emphasizing the user U having a small size bymaking the outline L0 of the user U having the small size thicker thanthe outline L1 of the user U having a large size. In this way, it iseasier to find the user in the captured image by increasing the degreeof emphasizing the user to the same extent to which the size of the userin the captured image decreases.

In addition, the user U may be hidden behind other objects in thecaptured image. For example, when a portion of the user or the entireuser is hidden and shown by other objects in the captured image, thedepiction change unit 132 may change the technique of emphasizing theuser. FIG. 22 is a diagram illustrating another example of the depictionchange function of the information processing device 10. As shown inFIG. 22, a portion of the user U is hidden by other subjects in thecaptured image. The depiction change unit 132 thus changes the techniqueof emphasizing the user U by adding the object Obj10 to a positionaround the user U in the captured image. The range of surrounding theuser U may be defined in advance.

Next, a flow of depiction change operations of the informationprocessing device 10 will be described. FIG. 23 is a flowchartillustrating the flow of the depiction change operations of theinformation processing device 10. As shown in FIG. 23, when the capturedimage is not acquired by the image acquisition unit 111 (“No” in stepS41), the flow is finished. On the other hand, when the captured imageis acquired by the image acquisition unit 111 (“Yes” in step S41), thesubject is recognized from the captured image by the recognition unit121 (step S42). The change detection unit 131 then decides whether ornot the change in state of the subject in the network service isdetected (step S43).

When the change in state of the subject in the network service is notdetected by the change detection unit 131 (“No” in step S43), the flowis finished. On the other hand, when the change in state of the subjectin the network service is detected by the change detection unit 131(“Yes” in step S43), the depiction change unit 132 changes the depictionof the subject (step S44). The display control unit 160 controls displayof the display image that may be obtained by the depiction change unit132 that changes the depiction (step S45), and the flow is finished.

As described above, according to the depiction change function of theinformation processing device 10, the change in state in the networkservice of the subject recognized from the captured image is detected.In addition, when the change in state is detected, the depiction of thesubject shown in the captured image is changed. According to suchcontrol, the change in state in the network service of the subject shownin the captured image is expected to be easily found.

4. CONCLUSION

As described above, according to the user specification function of theinformation processing device 10 of the present embodiment, theinformation processing device 10 including the image acquisition unit111 that acquires the captured image, the parameter acquisition unit 112that acquires the parameter associated with the user of the portableterminal 20, the specification unit 114 that specifies the user from thecaptured image based on the parameter, and the display control unit 160that control the portable terminal such that information indicating themovement of the user necessary to specify the user is displayed in theportable terminal 20 when the user is not specified by the specificationunit 114 is provided.

According to the configuration described above, when the user is notspecified by the specification unit 114, the information indicating themovement of the user necessary to specify the user is displayed by theportable terminal 20, and the user who has viewed the information canmove so as to specify the user from the captured image. Accordingly, theuser is expected to be easily specified from the captured image.

In addition, according to the composition decision function of theinformation processing device 10 of the present embodiment, theinformation processing device 10 including the image acquisition unit111 that acquires the captured image, the composition decision unit 125that decides the composition based on information indicating thedestination of the user and the recognition result of the user based onthe captured image, and the image determination unit 126 that determinesthe display image based on the composition decided by the compositiondecision unit 125 is provided.

According to such a configuration, the composition is decided based onthe information indicating the destination of the user and therecognition result of the user based on the captured image, and thedisplay image is determined based on the composition. For example, thecomposition is decided based on information indicating the user positionand information indicating the destination by the composition decisionunit 125. Accordingly, the display image having higher convenience isexpected to be determined for the user.

In addition, according to the depiction change function of theinformation processing device 10 of the present embodiment, theinformation processing device 10 including the image acquisition unit111 that acquires the captured image, the change detection unit 131 thatdetects the change in state in the network service of the subjectrecognized from the captured image, and the depiction change unit 132that changes the depiction of the subject shown in the captured imagewhen the change in state is detected by the change detection unit 131 isprovided.

According to such a configuration, when the change in state of thesubject in the network service is detected, the depiction of the subjectshown in the captured image is changed. It is possible for the user whohas viewed the captured image to easily find the change in state of thesubject in the network service from the captured image.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

In addition, for example, an example in which the information processingdevice 10 includes all of the processing control unit 100 and thedisplay control unit 160 has been described. However, a device otherthan the information processing device 10 may have a portion of theblocks or the entire blocks. For example, a server may have a portion ofthe processing control unit 100 and the display control unit 160 and theportable terminal 20 may have the remaining portion. In this way, thetechnique of the present disclosure can also be applied to cloudcomputing.

In addition, it is not necessary for each step in operations of theinformation processing device 10 of the present specification to beprocessed in a time series in the order described in the flowchart. Forexample, each step in the operations of the information processingdevice 10 may be processed in an order different from the orderdescribed in the flowchart, or may be processed in parallel.

In addition, it is possible to create a computer program causinghardware such as a CPU, a ROM, and a RAM embedded in the informationprocessing device 10 to exert the same function as the configuration ofthe information processing device 10 described above. In addition, astorage medium in which the computer program is stored is also provided.

Additionally, the present technology may also be configured as below.

(1) An information processing device including an image acquisition unitconfigured to receive an image; a recognition unit configured to acquirea recognition result of a user based on the received image, wherein therecognition result includes a position of the user, the user beingassociated with a display terminal; an image determination unitconfigured to determine an object based on the recognition result; and adisplay control unit configured to control display of the object on thedisplay terminal.

(2) The information processing device according to (1), furtherincluding a composition decision unit configured to determine acomposition of content for display on the display terminal based on therecognition result.

(3) The information processing device according to (2), wherein thecomposition decision unit is configured to determine the composition ofcontent based on information indicating a destination of the user.

(4) The information processing device according to (2), wherein thecomposition decision unit is configured to determine the composition ofcontent based on a detected direction of movement of the user.

(5) The information processing device according to (1), furtherincluding an information addition unit configured to add to the receivedimage an indication of a boundary of a region capable of being captured.

(6) The information processing device according to (1), wherein theimage acquisition unit is configured to capture the image.

(7) The information processing device according to (1), wherein theimage acquisition unit is configured to receive the image from anexternal device.

(8) The information processing device according to (1), wherein therecognition result includes an identity of the user.

(9) The information processing device according to (1), wherein therecognition result includes personal information associated with theuser.

(10) The information processing device according to (1), wherein theobject is not part of the received image.

(11) The information processing device according to (1), wherein theobject is a message.

(12) The information processing device according to (1), wherein theobject provides information associated with a destination of the user.

(13) The information processing device according to (1), wherein thedisplay control unit controls display of the received image on thedisplay terminal.

(14) A method for processing information, the method including receivingan image; acquiring a recognition result of a user based on the receivedimage, wherein the recognition result includes a position of the user,the user being associated with a display terminal; determining an objectbased on the recognition result; and controlling display of the receivedimage and the object on the display terminal.

(15) The method according to (14), wherein the recognition resultincludes an identity of the user.

(16) The method according to (14), wherein the recognition resultincludes personal information associated with the user.

(17) The method according to (14), wherein the object is not part of thereceived image.

(18) The method according to (14), wherein the object is a message.

(19) The method according to (14), wherein the object providesinformation associated with a destination of the user.

(20) A system including a display terminal associated with a user; aimage capturing device configured to capture an image; and aninformation processing device coupled to the display terminal and theimage capturing device, the information processing device including animage acquisition unit configured to receive the image from the imagecapturing device; a recognition unit configured to acquire a recognitionresult of the user based on the received image, wherein the recognitionresult includes a position of the user; an image determination unitconfigured to determine an object based on the recognition result; and adisplay control unit configured to control display of the received imageand the object on the display terminal.

(21) An information processing device including:

an image acquisition unit configured to acquire a captured image;

a composition decision unit configured to decide a composition based ona recognition result of a user based on the captured image andinformation indicating a destination of the user; and

an image determination unit configured to determine a display imagebased on the composition decided by the composition decision unit.

(22) The information processing device according to (21), furtherincluding: a display control unit configured to control the displayimage determined by the image determination unit to be displayed in aportable terminal.

(23) The information processing device according to (21) or (22),wherein, when a position of the user is recognized as the recognitionresult, the composition decision unit decides the composition based on adirection of the destination of the user using the position of the useras a reference.

(24) The information processing device according to (21) or (22),further including: a mode setting unit configured to set any one of afirst mode in which the composition is decided based on a direction ofmovement of the user and a second mode in which the composition isdecided based on a direction of the destination of the user using aposition of the user as a reference,

wherein the composition decision unit decides the composition inaccordance with the mode set by the mode setting unit.

(25) The information processing device according to any one of (21) to(4), wherein, when a position of the user is recognized as therecognition result, the composition decision unit decides thecomposition in which the user is present at a center or a position apredetermined distance away from the center.

(26) The information processing device according to any one of (21) to(25), wherein, when captured images provided from a respective pluralityof imaging devices are acquired by the image acquisition unit, the imagedetermination unit determines a captured image selected among thecaptured images provided from the respective plurality of imagingdevices as the display image.

(27) The information processing device according to any one of (21) to(25), wherein the image determination unit determines a capturingdirection of an imaging device that provides a captured image, anddetermines a captured image captured in the determined capturingdirection as the display image.

(28) The information processing device according to any one of (21) to(25), wherein the image determination unit determines an angle of viewof an imaging device that provides a captured image, and determines acaptured image captured at the determined angle of view as the displayimage.

(29) The information processing device according to any one of (21) to(25), wherein the image determination unit determines a region selectedfrom the captured image as the display image.

(30) The information processing device according to any one of (21) to

(25), wherein the image determination unit determines the display imagebased on detection data other than the composition when the imagedetermination unit is incapable of determining the display image basedon the composition decided by the composition decision unit.

(31) The information processing device according to (21) or (22),wherein, when the captured image is not acquired by the imageacquisition unit, the image determination unit determines an imageaccording to the information indicating the destination of the user anda position of the user detected based on data other than the capturedimage as the display image.

(32) The information processing device according to any one of (21) to(30), further including:

a recognition unit configured to obtain the recognition result byrecognizing the user based on the captured image.

(33) The information processing device according to any one of (21) to(32), further including:

a condition decision unit configured to decide whether or not acondition for capturing the user is prepared; and

an information addition unit configured to add information indicatingthat the condition is not prepared to the display image when thecondition decision unit decides that the condition is not prepared.

(34) An information processing method including:

acquiring a captured image;

deciding a composition based on a recognition result of a user based onthe captured image and information indicating a destination of the user;and

determining a display image based on the composition.

(35) A program for causing a computer to function as an information

processing device, the information processing device including:

an image acquisition unit configured to acquire a captured image;

a composition decision unit configured to decide a composition based ona recognition result of a user based on the captured image andinformation indicating a destination of the user; and

an image determination unit configured to determine a display imagebased on the composition decided by the composition decision unit.

REFERENCE SIGNS LIST

-   1 Information processing system-   10 Information processing device-   20 Portable terminal-   30 Imaging device-   40 Network-   50 Storage unit-   100 Processing control unit-   111 Image acquisition unit-   112 Parameter acquisition unit-   113 Candidate extraction unit-   114 Specification unit-   115 Authentication unit-   121 Recognition unit-   122 Condition decision unit-   123 Information addition unit-   124 Mode setting unit-   125 Composition decision unit-   126 Image determination unit-   131 Change detection unit-   132 Depiction change unit-   160 Display control unit

What is claimed is:
 1. An information processing system, comprising: aportable terminal including a display; a mobile object including animaging device configured to capture an image, and an actuator to movethe mobile object; and circuitry configured to: determine a position ofa specific subject related to the portable terminal based on thecaptured image; send the captured image and information on thedetermined position of the specific subject to the portable terminal;determine a first virtual object based on the determined position of thespecific subject; and control the display to display the first virtualobject over the captured image to indicate the determined position ofthe specific subject.
 2. The information processing system according toclaim 1, the portable terminal includes a touch panel configured toreceive a touch operation and is configured to send, to the circuitry,data of a position of the touch operation in the captured image, and thecircuitry is configured to determine the position of the subject relatedto the portable terminal based on the position of the touch operation inthe captured image.
 3. The information processing system according toclaim 1, wherein the circuitry is configured to: determine whether thereis a first subject performing a larger movement than a movement of asecond subject in the captured image; and determine the position of thespecific subject based on a position of the first subject, when it isdetermined that there is the first subject performing the largermovement in the captured image.
 4. The information processing systemaccording to claim 1, wherein the circuitry is configured to: determinea travelling direction of the specific subject based on the capturedimage; and change a composition of the captured image based on thedetermined position of the specific subject and the determinedtravelling direction of the specific subject.
 5. The informationprocessing system according to claim 4, wherein the imaging device is amovable camera configured to change a capturing direction in at leastone of a pan direction or a tilt direction, and the circuitry isconfigured to actuate the movable camera to change the composition ofthe captured image, based on the determined position of the specificsubject and the determined travelling direction of the specific subject.6. The information processing system according to claim 4, wherein thecircuitry is configured to change the composition of the captured imagesuch that a first displayed region of the captured image correspondingto the travelling direction becomes larger than a second displayedregion of the captured image corresponding to a direction opposite tothe traveling direction.
 7. The information processing system accordingto claim 1, wherein the circuitry is configured to: determine whetherthe specific subject is hidden behind at least one subject differentfrom the specific subject in the captured image; and control the displayto display a second virtual object different from the first virtualobject to emphasize the position of the specific subject in the capturedimage.
 8. The information processing system according to claim 1,wherein the circuitry is configured to control the imaging device tozoom-out the position of the specific subject after the position of thespecific subject is zoomed-in when the position of the specific subjectis determined.
 9. The information processing system according to claim1, wherein the first virtual object is a message object includinginstruction to move the specific subject.
 10. A method for controllingan information processing system including a portable terminal and amobile object, the method comprising: connecting the portable terminalwith the mobile object wirelessly; determining a position of a specificsubject related to the portable terminal based on an image captured bythe mobile object; sending the captured image and information on thedetermined position of the specific subject to the portable terminal;determining a virtual object based on the determined position of thespecific subject; and controlling a display of the portable terminal todisplay the virtual object over the captured image to indicate thedetermined position of the specific subject.
 11. At least onenon-transitory computer-readable medium encoded with instructions which,when executed by at least one of a portable terminal, a mobile object,and a network server, causes a combination of the portable terminal andthe mobile object to: connect the portable terminal with the mobileobject wirelessly; determine a position of a specific subject related tothe portable terminal based on an image captured by the mobile object;send the captured image and information on the determined position ofthe specific subject to the portable terminal; determine a virtualobject based on the determined position of the specific subject; andcontrol a display of the portable terminal to display the virtual objectover the captured image to indicate the determined position of thespecific subject.